Decison Boundary and hyperbolic unfolding¶

Decision boundary of a neural network¶

Dataset with two classes

Train a neural network on the binary classification task

In [197]:
# define fully connected NN architecture
circle_detect = Net(0, [2,10,20])


# Train neural net on binary classification task
_ = train_classification_nn(circle_detect, data, label, n_epochs=15)
epoch train_loss valid_loss accuracy time
0 0.734842 0.725282 0.405000 00:00
1 0.690313 0.658216 0.366000 00:00
2 0.592282 0.542293 0.932000 00:01
3 0.476480 0.434061 0.979000 00:00
4 0.394097 0.368978 1.000000 00:00
5 0.351687 0.340134 1.000000 00:01
6 0.332897 0.327614 1.000000 00:01
7 0.323875 0.321176 1.000000 00:01
8 0.319400 0.318098 1.000000 00:01
9 0.317187 0.316501 1.000000 00:00
10 0.316008 0.315628 1.000000 00:00
11 0.315367 0.315155 1.000000 00:00
12 0.315033 0.314920 1.000000 00:00
13 0.314886 0.314832 1.000000 00:00
14 0.314844 0.314819 1.000000 00:00

Contour plot of the neural network function

The descion boundray is $\left\{x\in \mathbb R^2:\ nn(x)=\frac 12\right\}$, i.e. all points where the output of the neural network is ambiguous.

Ways to compute the decision boundary¶

First Approach: Sampling algorithm¶

  • Using methods described in "On The Topological Expressive Power of Neural Networks", brought to our attention by Stefania at the last AppTop Meet-up.
  • Implementation by Matteo.

Gradient flow method¶

  • Pushing point to the decision boundary.
  • Using automatic differentiation of neural networks.
  • Fully implemented in Pytorch hence it can be strong acceleration via GPUs.

Sample points $x_i$ with gradients $-\nabla_{x_i} \left(nn(x_i)-\frac 12\right)^2$.

After 50 iterations and a filtration with respect to $|nn(x_i)-0.5|>0.01$ we end up with the red points:

Comparision with the previously plotted decision boundary.

Hyperbolic unfolding¶

  • Conformal transformation the euclidian metric with factor $1/(nn-\frac 12)$.
  • The resulting metric space is (Gromov-)hyperbolic with the decision boundary as boundary at infinity.
  • Geodesic ray converge to the boundary at infinity.
  • We use numerical methods for solving geodesic equations (second order differential equations).

Computed decision boundary with dataset¶

Entanglement problem for tori¶

Decision boundary for entangled tori¶

Decision boundary for unentangled tori¶